Creating Multi-Level Skill Hierarchies in Reinforcement Learning

What is a useful skill hierarchy for an autonomous agent? We propose an answer based on the graphical structure of an agent's interaction with its environment. Our approach uses hierarchical graph partitioning to expose the structure of the graph at varying timescales, producing a skill hierarchy with multiple levels of abstraction. At each level of the hierarchy, skills move the agent between regions of the state space that are well connected within themselves but weakly connected to each other. We illustrate the utility of the proposed skill hierarchy in a wide variety of domains in the context of reinforcement learning

Learning Structured Preferences

Learning the preferences of other people is crucial for predict- ing future behavior. Both children and adults make inferences about others’ preferences from sparse data and in situations where the preferences have complex internal structures. We present a computational model of learning structured prefer- ences which integrates Bayesian inference and utility-based models of preference from economics. We experimentally test this model with adult participants, and compare the model to alternative heuristic models

Si microwire-array solar cells

Si microwire-array solar cells with Air Mass 1.5 Global conversion efficiencies of up to 7.9% have been fabricated using an active volume of Si equivalent to a 4 μm thick Si wafer. These solar cells exhibited open-circuit voltages of 500 mV, short-circuit current densities (J_(sc)) of up to 24 mA cm^(-2), and fill factors >65% and employed Al_2O_3 dielectric particles that scattered light incident in the space between the wires, a Ag back reflector that prevented the escape of incident illumination from the back surface of the solar cell, and an a-SiN_x:H passivation/anti-reflection layer. Wire-array solar cells without some or all of these design features were also fabricated to demonstrate the importance of the light-trapping elements in achieving a high J_(sc). Scanning photocurrent microscopy images of the microwire-array solar cells revealed that the higher J_(sc) of the most advanced cell design resulted from an increased absorption of light incident in the space between the wires. Spectral response measurements further revealed that solar cells with light-trapping elements exhibited improved red and infrared response, as compared to solar cells without light-trapping elements

Help or hinder: Bayesian models of social goal inference

Everyday social interactions are heavily influenced by our snap judgments about others’ goals. Even young infants can infer the goals of intentional agents from observing how they interact with objects and other agents in their environment: e.g., that one agent is ‘helping’ or ‘hindering’ another’s attempt to get up a hill or open a box. We propose a model for how people can infer these social goals from actions, based on inverse planning in multiagent Markov decision problems (MDPs). The model infers the goal most likely to be driving an agent’s behavior by assuming the agent acts approximately rationally given environmental constraints and its model of other agents present. We also present behavioral evidence in support of this model over a simpler, perceptual cue-based alternative.United States. Army Research Office (ARO MURI grant W911NF-08-1-0242)United States. Air Force Office of Scientific Research (MURI grant FA9550-07-1-0075)National Science Foundation (U.S.) (Graduate Research Fellowship)James S. McDonnell Foundation (Collaborative Interdisciplinary Grant on Causal Reasoning

The Very Low Albedo of WASP-12b From Spectral Eclipse Observations with Hubble\textit{Hubble}

We present an optical eclipse observation of the hot Jupiter WASP-12b using the Space Telescope Imaging Spectrograph on board the Hubble Space Telescope. These spectra allow us to place an upper limit of $A_g < 0.064$ (97.5% confidence level) on the planet's white light geometric albedo across 290--570 nm. Using six wavelength bins across the same wavelength range also produces stringent limits on the geometric albedo for all bins. However, our uncertainties in eclipse depth are $\sim$40% greater than the Poisson limit and may be limited by the intrinsic variability of the Sun-like host star --- the solar luminosity is known to vary at the $10^{-4}$ level on a timescale of minutes. We use our eclipse depth limits to test two previously suggested atmospheric models for this planet: Mie scattering from an aluminum-oxide haze or cloud-free Rayleigh scattering. Our stringent nondetection rules out both models and is consistent with thermal emission plus weak Rayleigh scattering from atomic hydrogen and helium. Our results are in stark contrast with those for the much cooler HD 189733b, the only other hot Jupiter with spectrally resolved reflected light observations; those data showed an increase in albedo with decreasing wavelength. The fact that the first two exoplanets with optical albedo spectra exhibit significant differences demonstrates the importance of spectrally resolved reflected light observations and highlights the great diversity among hot Jupiters.Comment: 8 pages, 4 figures, 1 table, published in ApJL, in pres

Survival outcome according to KRAS mutation status in newly diagnosed patients with stage IV non-small cell lung cancer treated with platinum doublet chemotherapy

INTRODUCTION: Mutations (MT) of the KRAS gene are the most common mutation in non-small cell lung cancer (NSCLC), seen in about 20-25% of all adenocarcinomas. Effect of KRAS MT on response to cytotoxic chemotherapy is unclear. METHODS: We undertook a single-institution retrospective analysis of 93 consecutive patients with stage IV NSCLC adenocarcinoma with known KRAS and EGFR MT status to determine the association of KRAS MT with survival. All patients were treated between January 1, 2008 and December 31, 2011 with standard platinum based chemotherapy at the University of Pennsylvania. Overall and progression free survival were analyzed using Kaplan-Meier and Cox proportional hazard methods. RESULTS: All patients in this series received platinum doublet chemotherapy, and 42 (45%) received bevacizumab. Overall survival and progression free survival for patients with KRAS MT was no worse than for patients with wild type KRAS. Median overall survival for patients with KRAS MT was 19 months (mo) vs. 15.6 mo for KRAS WT, p = 0.34, and progression-free survival was 6.2 mo in patients with KRAS MT vs. 7mo in patients with KRAS WT, p = 0.51. In multivariable analysis including age, race, gender, and ECOG PS, KRAS MT was not associated with overall survival (HR 1.12, 95% CI 0.58-2.16, p = 0.74) or progression free survival (HR 0.80, 95% CI 0.48-1.34, p = 41). Of note, receipt of bevacizumab was associated with improved overall survival only in KRAS WT patients (HR 0.34, p = 0.01). CONCLUSIONS: KRAS MT are not associated with inferior progression-free and overall survival in advanced NSCLC patients treated with standard first-line platinum-based chemotherapy

Spectral Energy Distributions of Local Luminous And Ultraluminous Infrared Galaxies

Luminous and ultraluminous infrared galaxies ((U)LIRGs) are the most extreme star forming galaxies in the universe. The local (U)LIRGs provide a unique opportunity to study their multi-wavelength properties in detail for comparison to their more numerous counterparts at high redshifts. We present common large aperture photometry at radio through X-ray wavelengths, and spectral energy distributions (SEDs) for a sample of 53 nearby LIRGs and 11 ULIRGs spanning log (LIR/Lsun) = 11.14-12.57 from the flux-limited Great Observatories All-sky LIRG Survey (GOALS). The SEDs for all objects are similar in that they show a broad, thermal stellar peak and a dominant FIR thermal dust peak, where nuLnu(60um) / nuLnu(V) increases from ~2-30 with increasing LIR. When normalized at IRAS-60um, the largest range in the luminosity ratio, R(lambda)=log[nuLnu(lambda)/nuLnu(60um)] observed over the full sample is seen in the Hard X-rays (HX=2-10 keV). A small range is found in the Radio (1.4GHz), where the mean ratio is largest. Total infrared luminosities, LIR(8-1000um), dust temperatures, and dust masses were computed from fitting thermal dust emission modified blackbodies to the mid-infrared (MIR) through submillimeter SEDs. The new results reflect an overall ~0.02 dex lower luminosity than the original IRAS values. Total stellar masses were computed by fitting stellar population synthesis models to the observed near-infrared (NIR) through ultraviolet (UV) SEDs. Mean stellar masses are found to be log(M/Msun) = 10.79+/-0.40. Star formation rates have been determined from the infrared (SFR_IR~45Msun/yr) and from the monochromatic UV luminosities (SFR_UV~1.3Msun/yr), respectively. Multiwavelength AGN indicators have be used to select putative AGN: about 60% of the ULIRGs would have been classified as an AGN by at least one of the selection criteria.Comment: 39 pages, including 12 figures and 11 tables; accepted for publication in ApJ

Energy-Conversion Properties of Vapor-Liquid-Solid–Grown Silicon Wire-Array Photocathodes

Silicon wire arrays, though attractive materials for use in photovoltaics and as photocathodes for hydrogen generation, have to date exhibited poor performance. Using a copper-catalyzed, vapor-liquid-solid–growth process, SiCl_4 and BCl_3 were used to grow ordered arrays of crystalline p-type silicon (p-Si) microwires on p^+-Si(111) substrates. When these wire arrays were used as photocathodes in contact with an aqueous methyl viologen^(2+/+) electrolyte, energy-conversion efficiencies of up to 3% were observed for monochromatic 808-nanometer light at fluxes comparable to solar illumination, despite an external quantum yield at short circuit of only 0.2. Internal quantum yields were at least 0.7, demonstrating that the measured photocurrents were limited by light absorption in the wire arrays, which filled only 4% of the incident optical plane in our test devices. The inherent performance of these wires thus conceptually allows the development of efficient photovoltaic and photoelectrochemical energy-conversion devices based on a radial junction platform

Targeting the LOX/hypoxia axis reverses many of the features that make pancreatic cancer deadly: inhibition of LOX abrogates metastasis and enhances drug efficacy

Pancreatic ductal adenocarcinoma (PDAC) is one of the leading causes of cancer‐related mortality. Despite significant advances made in the treatment of other cancers, current chemotherapies offer little survival benefit in this disease. Pancreaticoduodenectomy offers patients the possibility of a cure, but most will die of recurrent or metastatic disease. Hence, preventing metastatic disease in these patients would be of significant benefit. Using principal component analysis (PCA), we identified a LOX/hypoxia signature associated with poor patient survival in resectable patients. We found that LOX expression is upregulated in metastatic tumors from Pdx1‐Cre KrasG12D/+ Trp53R172H/+ (KPC) mice and that inhibition of LOX in these mice suppressed metastasis. Mechanistically, LOX inhibition suppressed both migration and invasion of KPC cells. LOX inhibition also synergized with gemcitabine to kill tumors and significantly prolonged tumor‐free survival in KPC mice with early‐stage tumors. This was associated with stromal alterations, including increased vasculature and decreased fibrillar collagen, and increased infiltration of macrophages and neutrophils into tumors. Therefore, LOX inhibition is able to reverse many of the features that make PDAC inherently refractory to conventional therapies and targeting LOX could improve outcome in surgically resectable disease